<!DOCTYPE html>
<html>

<head>
	<meta charset="utf-8">
	<meta name="viewport" content="width=device-width, initial-scale=1, maximum-scale=1">
	<meta name="theme-color" content="#33474d">
	<title>LVS原理详解及部署之三：手动部署LVS | 失落的乐章</title>
	<link rel="stylesheet" href="/css/style.css" />
	
      <link rel="alternate" href="/atom.xml" title="失落的乐章" type="application/atom+xml">
    
</head>

<body>

	<header class="header">
		<nav class="header__nav">
			
				<a href="/archives" class="header__link">Archive</a>
			
				<a href="/tags" class="header__link">Tags</a>
			
				<a href="/atom.xml" class="header__link">RSS</a>
			
		</nav>
		<h1 class="header__title"><a href="/">失落的乐章</a></h1>
		<h2 class="header__subtitle">技术面前，永远都是学生。</h2>
	</header>

	<main>
		<article>
	
		<h1>LVS原理详解及部署之三：手动部署LVS</h1>
	
	<div class="article__infos">
		<span class="article__date">2017-10-12</span><br />
		
		
			<span class="article__tags">
			  	<a class="article__tag-link" href="/tags/LVS/">LVS</a>
			</span>
		
	</div>

	

	
		<h2 id="一、环境需求-amp-安装LVS软件"><a href="#一、环境需求-amp-安装LVS软件" class="headerlink" title="一、环境需求&amp;安装LVS软件"></a>一、环境需求&amp;安装LVS软件</h2><h3 id="环境准备：三台虚拟机"><a href="#环境准备：三台虚拟机" class="headerlink" title="环境准备：三台虚拟机"></a>环境准备：三台虚拟机</h3><p>&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;此环境是针对内部服务的LVS架构，如数据库，缓存，共享存储等业务。</p>
<table>
<thead>
<tr>
<th>虚拟机角色</th>
<th>IP地址</th>
<th>备注</th>
</tr>
</thead>
<tbody>
<tr>
<td>LVS负载均衡器</td>
<td>192.168.41.181</td>
<td>VIP地址：192.168.40.17</td>
</tr>
<tr>
<td>http服务器</td>
<td>RS1192.168.41.31</td>
<td>&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;</td>
</tr>
<tr>
<td>http服务器</td>
<td>RS2192.168.41.33</td>
<td>&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;</td>
</tr>
</tbody>
</table>
<h3 id="安装LVS软件"><a href="#安装LVS软件" class="headerlink" title="安装LVS软件"></a>安装LVS软件</h3><ol>
<li><p>在安装LVS软件之前，先确定两条HTTPserver是能够正常访问的。</p>
</li>
<li><p>下载软件</p>
</li>
</ol>
<figure class="highlight bash"><table><tr><td class="gutter"><pre><div class="line">1</div></pre></td><td class="code"><pre><div class="line">wget http://www.linuxvirtualserver.org/software/kernel-2.6/ipvsadm-1.24.tar.gz</div></pre></td></tr></table></figure>
<p>&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;这里使用的2.4版本，并且注意内核是2.6版本的，如果版本是6.X版本的话，那么可以使用2.6版本</p>
<ol>
<li>编译安装</li>
</ol>
<p>&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;需要创建一个软连接：ln -s /usr/src/kernels/<strong>2.6.18-238.el5-i686</strong> /usr/src/linux</p>
<p>&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;此处粗体根据自己的系统来进行定义，可以使用tab键来补齐。</p>
<figure class="highlight bash"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div></pre></td><td class="code"><pre><div class="line">tar -zxf ipvsadm-1.24.tar.gz</div><div class="line"><span class="built_in">cd</span> ipvsadm-1.24</div><div class="line">make</div><div class="line">make install</div><div class="line">lsmod |grep ip_vs</div><div class="line">ipvsadm</div></pre></td></tr></table></figure>
<p>&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;因为此时系统还没有把ipvs模块加载进系统，需要我们执行ipvsadm命令才会加载进去,或者modprobe ip_vs。</p>
<figure class="highlight bash"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div></pre></td><td class="code"><pre><div class="line">[root@localhost ipvsadm-1.24]<span class="comment"># lsmod |grep ip_vs</span></div><div class="line">ip_vs_rr 6081 1</div><div class="line">ip_vs 78081 3 ip_vs_rr</div></pre></td></tr></table></figure>
<h2 id="二、手动配置LVS负载均衡器"><a href="#二、手动配置LVS负载均衡器" class="headerlink" title="二、手动配置LVS负载均衡器"></a>二、手动配置LVS负载均衡器</h2><p>&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;正常工作中是不会手动配置的，也不会使用脚本配置的。最终我们是通过配置文件生效的，结合keepalived来进行部署的。</p>
<ol>
<li>负载均衡器上配置VIP地址</li>
</ol>
<figure class="highlight bash"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div></pre></td><td class="code"><pre><div class="line">ifconfig eth0:1 192.168.40.17 netmask 255.255.254.0</div><div class="line">route add -host 192.168.40.17  dev eth0</div></pre></td></tr></table></figure>
<ol>
<li>ipvsadm添加LVS服务</li>
</ol>
<table>
<thead>
<tr>
<th>参数</th>
<th>参数说明</th>
</tr>
</thead>
<tbody>
<tr>
<td>-A</td>
<td>-A –add-service 添加一个带选项的虚拟服务。Add a virtual service. A serviceaddress is uniquely defined by a triplet: IP address, portnumber, and protocol. Alternatively a virtualservice may be defined by a firewall-mark.</td>
</tr>
<tr>
<td>-t</td>
<td>指定虚拟服务器的IP地址和端口</td>
</tr>
<tr>
<td>-s</td>
<td>-s,–scheduler scheduling-method 调度算法</td>
</tr>
<tr>
<td>-p</td>
<td>会话保持按秒计算</td>
</tr>
<tr>
<td>-a</td>
<td>-a在对应的VIP下添加RS节点</td>
</tr>
<tr>
<td>-g</td>
<td>指定此LVS的工作模式为-g -g为DR模式</td>
</tr>
<tr>
<td>-l</td>
<td>指定LVS的工作模式为-l -l为tunnel模式</td>
</tr>
<tr>
<td>-m</td>
<td>指定LVS的工作模式为NAT模式</td>
</tr>
<tr>
<td>-w</td>
<td>指定RS节点的权重</td>
</tr>
<tr>
<td>-D</td>
<td>删除虚拟服务.格式：ipvsadm-D -t</td>
<td>u</td>
<td>f service-address. Delete a virtual service, alongwith any associated real servers.</td>
</tr>
<tr>
<td>-C</td>
<td>-C, –clear Clear the virtual server table清空lvs原有的配置。</td>
</tr>
<tr>
<td>-set</td>
<td>设置tcp tcpfn udp 的连接超时时间（一般来说高并发的时候小一点点。</td>
</tr>
</tbody>
</table>
<figure class="highlight bash"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div></pre></td><td class="code"><pre><div class="line">ipvsadm  -C                                                <span class="comment">#请用LVS原有的配置</span></div><div class="line">ipvsadm -A -t 192.168.40.17:80 -s rr -p 20                 <span class="comment">#添加虚拟服务指定VIP</span></div><div class="line">ipvsadm -a -t 192.168.40.17:80 -r 192.168.41.31:80 -g -w 10<span class="comment">#针对虚拟服务添加RS节点</span></div><div class="line">ipvsadm -a -t 192.168.40.17:80 -r 192.168.41.33:80 -g -w 10</div><div class="line">ipvsadm -L -n                                              <span class="comment">#查看VIP和RS是否已经配置成功。</span></div></pre></td></tr></table></figure>
<figure class="highlight bash"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div></pre></td><td class="code"><pre><div class="line">[root@localhost ~]<span class="comment"># ipvsadm -L -n</span></div><div class="line">IP Virtual Server version 1.2.1 (size=4096)</div><div class="line">Prot LocalAddress:Port Scheduler Flags</div><div class="line"> -&gt; RemoteAddress:Port Forward Weight ActiveConn InActConn</div><div class="line">TCP 192.168.40.17:80 rr persistent 20</div><div class="line"> -&gt; 192.168.41.33:80 Route 10 0 0 </div><div class="line"> -&gt; 192.168.41.31:80 Route 10 0 0</div></pre></td></tr></table></figure>
<p>&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;LB上删除虚拟服务</p>
<figure class="highlight bash"><table><tr><td class="gutter"><pre><div class="line">1</div></pre></td><td class="code"><pre><div class="line">ipvsadm -D -t 192.168.40.17:80</div></pre></td></tr></table></figure>
<h2 id="三、RS节点服务器手动配置"><a href="#三、RS节点服务器手动配置" class="headerlink" title="三、RS节点服务器手动配置"></a>三、RS节点服务器手动配置</h2><ol>
<li>添加lo端口的VIP&amp;路由</li>
</ol>
<figure class="highlight bash"><table><tr><td class="gutter"><pre><div class="line">1</div></pre></td><td class="code"><pre><div class="line">ifconfig lo 192.168.40.17 netmask 255.255.255.255</div></pre></td></tr></table></figure>
<p>&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;(由于RS的VIP不是用来通讯，并且这里一定要设置24位掩码）</p>
<figure class="highlight bash"><table><tr><td class="gutter"><pre><div class="line">1</div></pre></td><td class="code"><pre><div class="line">route add -host 192.168.40.17  dev lo</div></pre></td></tr></table></figure>
<ol>
<li>ARP抑制</li>
</ol>
<figure class="highlight bash"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div></pre></td><td class="code"><pre><div class="line"><span class="built_in">echo</span> <span class="string">"1"</span>&gt;/proc/sys/net/ipv4/conf/lo/arp_ignore</div><div class="line"><span class="built_in">echo</span> <span class="string">"2"</span>&gt;/proc/sys/net/ipv4/conf/lo/arp_announce </div><div class="line"><span class="built_in">echo</span> <span class="string">"1"</span>&gt;/proc/sys/net/ipv4/conf/all/arp_announce</div><div class="line"><span class="built_in">echo</span> <span class="string">"2"</span>&gt;/proc/sys/net/ipv4/conf/all/arp_announce</div></pre></td></tr></table></figure>
<h2 id="四、测试LVS是否生效"><a href="#四、测试LVS是否生效" class="headerlink" title="四、测试LVS是否生效"></a>四、测试LVS是否生效</h2><p>&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;在LB上面输入命令ipvsadm -L -n就能够查看LB上面的会话分配。在前面加上watch可以动态的查看ipvsadm的会话分配。watch ipvsadm -L -n.</p>
<ol>
<li>测试RS节点是否正常访问</li>
<li>测试从LB能否正常访问RS</li>
<li>测试客户端能否正常访问VIP</li>
</ol>
<p>&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;在测试的时候可以先把防火墙关闭掉，一般按照这样配置就能够实现LVS的负载均衡了。</p>
<p>&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;至此我们的LVS DR模式负载均衡已经配置完成了。至于不同的调度算法啊-s  不同的工作模式-g(DR) -l(TUNNEL) -m(NAT)服务器端基本上没有什么差别。只是在客户端上有一定的差别。</p>
<h3 id="NAT模式："><a href="#NAT模式：" class="headerlink" title="NAT模式："></a>NAT模式：</h3><p>&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;客户端同样需要配置VIP，进行ARP抑制，并且要服务器端开启内核转发功能，配置LB的DIP(内网IP地址）作为默认网关。</p>
<p>&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;开启内核转发功能：</p>
<figure class="highlight bash"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div></pre></td><td class="code"><pre><div class="line">vim /etc/sysctl</div><div class="line"></div><div class="line">net.ipv4.ip_forword = 1</div><div class="line">route add default gw  192.168.41.181</div></pre></td></tr></table></figure>
<h3 id="Tunnel模式："><a href="#Tunnel模式：" class="headerlink" title="Tunnel模式："></a>Tunnel模式：</h3><p>&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;客户端需要先开启Tunnel协议支持。</p>
<figure class="highlight bash"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div></pre></td><td class="code"><pre><div class="line">/sbin/modprobe ipip</div><div class="line">/sbin/route add –host 192.168.40.17 devtun1</div><div class="line"><span class="built_in">echo</span>”1”&gt;/proc/sys/net/ipv4/conf/tun1/arp-ignore</div><div class="line"><span class="built_in">echo</span>”2”&gt;/proc/sys/net/ipv4/conf/tun1/arp_announce</div><div class="line"><span class="built_in">echo</span>”0” &gt;/proc/sys/net/ipv4/conf/tun1/rp_filter</div><div class="line"><span class="built_in">echo</span>”1” &gt;/proc/sys/net/ipv4/conf/tun1/forwarding</div><div class="line"><span class="built_in">echo</span>”1” &gt;/proc/sys/net/ipv4/conf/all/arp_ignore</div><div class="line"><span class="built_in">echo</span>”2” &gt;/proc/sys/net/ipv4/conf/all/ arp_announce</div></pre></td></tr></table></figure>
<h2 id="五、部署成功后的另一些问题"><a href="#五、部署成功后的另一些问题" class="headerlink" title="五、部署成功后的另一些问题"></a>五、部署成功后的另一些问题</h2><ol>
<li>当我们的RS节点出现问题，LB如何知道。如果不知道是会把会话连接接续转发到RS上面。</li>
<li>如果LB出现故障，那么整个网络就出现故障。</li>
</ol>
<p>&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;针对上面的1问题，我们就需要一种RS节点健康检查机制。定时的去检测RS是否正常，如果出现不正常那么就把这个RS从VIP服务里面删除掉。如果恢复正常了，就再把RS添加进来。针对2问题，我们可以另外再架设一台LB服务器，作为备LB服务器。那么当主LB出现故障，备LB服务器就会启动接管主LB服务器的工作，接管它的资源（IP地址，在网络中的角色身份等）</p>
<p>&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;而上面提到的这些我们就需要结合keepalived来完成。所以后续我们开始讲keepalived+lvs结合适用。完成RS节点健康检查和LVS的高可用性功能。</p>

	

	
		<span class="different-posts"><a href="/2017/10/12/LVS/10. LVS原理详解及部署之三：手动部署LVS/" onclick="window.history.go(-1); return false;">⬅️ Go back </a></span>

	

</article>

	</main>

	<footer class="footer">
	<div class="footer-content">
		
	      <div class="footer__element">
	<p>Hi there, <br />welcome to my Blog glad you found it. Have a look around, will you?</p>
</div>

	    
	      <div class="footer__element">
	<h5>Check out</h5>
	<ul class="footer-links">
		<li class="footer-links__link"><a href="/archives">Archive</a></li>
		
		  <li class="footer-links__link"><a href="/atom.xml">RSS</a></li>
	    
		<li class="footer-links__link"><a href="/about">about page</a></li>
		<li class="footer-links__link"><a href="/tags">Tags</a></li>
		<li class="footer-links__link"><a href="/categories">Categories</a></li>
	</ul>
</div>

	    

		<div class="footer-credit">
			<span>© 2017 失落的乐章 | Powered by <a href="https://hexo.io/">Hexo</a> | Theme <a href="https://github.com/HoverBaum/meilidu-hexo">MeiliDu</a></span>
		</div>

	</div>


</footer>



</body>

</html>
